Angular alignment of structures using moire patterns

ABSTRACT

This invention uses an optical imaging system input to a computer to synthesize regularly spaced reference and test line patterns replicated from binarized edges taken from the optical images of structures to be aligned. The two patterns are then combined resulting in an interference pattern in which the spacing of moire fringes indicates the amount of movement of the structures required to bring them to a predetermined alignment. All of the procedures are observable on the interference display. These moire fringe pattern are a quantitative measure of angular alignment between the reference and test images and of the edges from the structures from which they were taken. An operator programmed and controlled template automatically processes moire edge alignment measurements. 
     An automatic template processor generates a test sequence template from the chosen areas for measurement by the operator. A template memory module having a library of test sequence templates allows the process to be run without the selection of areas to be measured by the operator. Process imaging and logic operations are utilized in generating the automatic test sequence template.

cross references to related Applications References is made to thefollowing co-pending applications of which the present application is acontinuation in part:

(a) Ser. No. 08/093,946 filed Jul. 21, 1993, now abandoned, entitled"ANGULAR ALIGNMENT BY EDGE DIFFRACTION MOIRE PATTERNS" in the name ofRobert W. Brandstetter and assigned to Grumman Corporation; and

(b) Ser. No. 08/212,595 filed Mar. 11, 1994, now abandoned, entitled"AUTOMATIC TEST TEMPLATE SYSTEM" in the names of Robert W. Brandstetterand Nils J. Fonneland, and assigned to Grumman Corporation.

FIELD OF THE INVENTION

The present invention relates to a computer controlled optical systemfor angular alignment of structures. More particularly, the inventionrelates to such a system in which an optical subsystem provides linedata (in image form) from the structures to a computer subsystem whereinmoirE patterns are generated from the line data to display angularalignment of the structures.

BACKGROUND OF THE INVENTION

In the manufacture of large structures such as aircraft, spacecraft,etc., where precision alignment is needed, it is known to employ opticalmetrology (including laser range finding and interferometrics) andphotogrammetry to determine the angular orientation of assemblies andsubassemblies in relation to each other. These processes can take anextended time, even days, to arrive at a result depending on therequired precision. Furthermore, these processes have often interferedwith work on or near the structure, even requiring that work be stoppedduring periods in which alignment measurements and alignment changes aremade.

There is, therefore, a need for a new and improved system for rapidangular alignment of structures.

OBJECTS OF THE INVENTION

It is a general object of the present invention to provide a method andapparatus for angular alignment of structures which will overcome theabove limitations and disadvantages.

It is a further general object of the present invention to provide asystem of the above character including an optical subsystem whichprovides images from the structures from which line date is derived andused in a computer subsystem to generate moire patterns which arecombined in an interference pattern which, when displayed, shows angularalignment.

It is a further general object of the present invention to provide asystem of the above character further in which the computer creates abinarized image of selected linear features of the structures from whichregularly spaced line patterns are derived which, when superimposed orinterfered, create moire interference patterns in which the fringespacing is an immediate indication of angular displacement of thestructures. The moire pattern is then observed as the structures aremoved into alignment and gives a critical report of the exact, properalignment.

It is another general object of the invention to provide a system of theabove character by which an optical images of linear features of thestructures are created, replicated into regularly spaced line patternsand logically combined to give an interference or moire fringe patternwhich is then displayed on a computer screen. The spacing is a measureof the angular misalignment.

If it is desired to obtain a predetermined angle (φ≠0) between thelinear features of the structures, one pattern is rotated through thepredetermined alignment angle by computer manipulation, and that rotatedpattern is then combined with the other to form a moire interferencepattern which now translates to a measurement of the amount ofmisalignment from that predetermined angle. The structures are thenangularly shifted into exact alignment to the angle φ by watching themoire interference pattern expand to the limit.

It is another general object of the present invention to provide systemof the above character for use in doing real time angular alignment ofstructural subassemblies and final assemblies, independent of x-ytranslation (shift invariant) of these assemblies.

It is a further object of the present invention to provide an automatedsystem for automatic test sequencing for a multiplicity of specifiededges of structures being aligned.

It is another object of the present invention to provide an automatedsystem of the above character capable of using a large library ofpre-planned automatic test procedures for moire alignment of structure.

It is another object of the present invention to provide an automatedsystem which allows the moire technique to automatically monitoralignment of structures located remotely from the operator.

It is a further object of the present invention to provide an automatedsystem that allows the real time alignment of large surfaces to highprecision.

It is still a further object of the invention to provide an automatedsystem whereby the process is not affected by translational errors inthe location of the production assembly when the operator originates orplaces automatic test templates which form part of the invention.

Another object of the invention is to provide an automated system whichcan be used for precision alignment of parts and assemblies withstraight or curved edges.

SUMMARY OF THE INVENTION

The present invention is based on the realization that optically formed,plan view images of the structures can be taken with a camera remotelylocated from the structures, i.e., above the structures, and deliveredas input to a computer programmed to generate, from these opticalimages, binarized line images from the linear part of the structuresthat it is desired to align. From these line images, linear repeatingpatterns of equally spaced parallel lines which follow contours ofselected object edges (simulating diffraction patterns) are generatedand logically combined for display on a computer screen readout to formreal time moire patterns showing the angular state of alignment betweenthe structures. The computer further is programmed to generate, ondemand, an angular shift of one pattern with respect to the other whichis selected to be the amount of the desired angular displacement betweenthe structures. Should the structures be aligned to that selected angle,the moire pattern will so show as an interference pattern having withinfinite fringe spacing, while misalignment will show as a plurality ofspaced parallel fringes the spacing (or frequency) of which isproportional to the misalignment. The structures can then be aligneduntil the fringes disappear, or, if solely measurement be sought, theangular displacement from the reference value inputted to the computercan be taken from a count or from the periodicity of the fringes.

As conceived, the edge transform was thought of as a Fraunhoffer edgediffraction pattern that could be modeled using a transform. However, itwas quickly determined that the edge transform is not limited to a modelor a few specific transform operators. In accordance with this inventionit is synthesized by any of a variety of processors or algorithms thatsatisfy the need to provide a replicated line pattern of equally spacedlines that are exactly parallel to the features (edges) of thestructures being aligned.

Thus, the edge alignment concept just stated makes use of two basicprinciples: (1) each structural feature, i.e., edge, can be transformedto a repetitive pattern of evenly spaced parallel lines (similar to thatgenerated by the classic diffraction pattern of monochromatic light froman edge) and (2) a moire fringe pattern results when two such repetitivepatterns are overlaid (in computer memory).

The apparatus of the invention further provides operator programmed andcontrolled template for automatic processing of moire edge alignmentmeasurements.

The apparatus includes a high resolution video camera and a moireprocessing computer connected to the video camera. A display is coupledto the moire processing computer. Automatic template processor is intwo-way communication with the moire processing computer. A light pen ormouse is connected to the automatic template processor and the displayand a template memory is in two-way connection to the automatic templateprocessor. An operator uses the light pen or mouse to selectivelyhighlight edge areas for moire alignment measurements. Alternatively, apredetermined test template sequence can be called up from the templatememory. The system then runs through a sequence of processes to producethe selected edges for moire alignment.

The first sequence of steps serve to produce a processed image from thetest image as viewed by the high resolution video camera. In order toproduce the processed image, the test image is thresholded, thenbinarized and then edged with the respective corresponding circuits toproduce a processed image.

The second sequence of steps is to produce a display template from thetest image. This is achieved by first marking up the test image with thelight pen or mouse. The marked up test image is then thresholded andthen binarized. After the step of binarizing, the binarized marked upimage is inverted to provide a display template.

Upon completion of the aforementioned sequence of steps, the processedimage and the display template are combined to produce selected edgesfor moire alignment.

These and other features and objects of the invention will now beexplained in detail by reference to the following detailed descriptionand claims when taken with the accompanying drawings, of which:

BRIEF DESCRIPTION TO THE DRAWINGS

FIG. 1 is a block diagram of the apparatus for angular alignmentconstructed in accordance with the present invention;

FIG. 2 is a reference image as developed by the apparatus of FIG. 1;

FIG. 3 is a test image as developed by the apparatus of FIG. 1;

FIG. 4 is a moire pattern from combined reference and test images asgenerated in the video display of the apparatus of FIG. 1;

FIG. 5 is an enlarged portion of the moire interference pattern of FIG.4;

FIG. 6 is a sketch showing angular relationships of the moireinterference pattern of FIG. 5;

FIG. 7 is a block diagram of a second apparatus adapted for automatedtest template use and constructed in accordance with the presentinvention;

FIG. 8 is a diagram of a basic first process of the present invention;

FIG. 9 is a diagram of a second process according to the invention; and

FIG. 10 is a diagram of a third process according to the invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to FIG. 1, there is shown an aircraft 10 as viewed from thetop, the aircraft resting on a production floor 12. The present systemwill be shown to achieve angular alignment of an aircraft surface and/oredge 13 relative to a reference axis, such as a reference line 14 takenfrom the fuselage centerline 15. In FIG. 1, a window 18 defines a wingedge for which alignment is to be checked relative to a reference line,for example the fuselage reference line 14. By virtue of the presentinvention, the angle of the wing edge 13 relative to the fuselagereference line may be measured, or during production the wing may beincrementally moved into place relative to the fuselage until a desiredalignment is achieved.

The scene is viewed by a video camera 20, preferably of high resolution,such as that manufactured by the Kodak company and marketed as a VidekMegaplus, or, less preferably, one with somewhat lower resolution suchas a generic RS170 CCD camera. Typically, the reference line 14 is, forexample, a fuselage centerline line rotated through an appropriate angleby a computer program, and, is thereafter replicated in the computer 46to form an image 41 of a first reference line pattern which is storedand interfered with a second image line pattern 51 replicated from asecond aircraft edge 13 so as to form a moire display indicative of therelative alignment between the inspected edge 13 and the derivedreference line 14. Combining patterns 41 and 51 results in a moirepattern 55 whose fringe spacing D is a direct measure of relativealignment between the derived reference line 14 and the test edge 13.

In order to store the images of the reference line 14 and inspected edge13, the output from the television camera 20 is input to a commercialframe grabber 32 which captures and processes the video. Acursor-controlled window selector 26 generates control signals for thedesired window for inspection, such as a portion of the forward wingedge 16 of aircraft 10--effectively creating a masked video output 34from the frame grabber. The masked video undergoes a digital transformin a processor 36 or equivalent means. The output 38 of the processor 36is input to a processor 40 which sets threshold detection to maximizethe signal-to-noise ratio of the input signal and also a binarizedimage. Note that FIG. 1 is meant to present the invention with itsvarious elements as discrete operations, e.g. processor 36, windowselector operator controls 26, monitor 24, and display 52. The functionsof the remaining components 32, 36, 40, and 46 could reside in a singleprocessor-computer operating with a custom software package or one witha modified source code such as the NIH application--IMAGE. The imagesare visually observed on a display 52. The processor 40 has anadditional output path, which is indicated in dotted lines to signifyits use during the storage of the reference image. Thus, the referenceimage is outputted from processor 40 and input 44 to a computer 46 orother memory medium. The window selector 26 establishes location datafor the reference edge image so that various reference edges or surfacesmay be addressed by corresponding windows relative to the aircraft 10.Window selection of the stored image occurs along control input 48 tothe computer 46.

Although the preceding discussion indicates a utilization of a visuallyinput reference, it is possible to store such a reference fromsimulation models.

Once the reference is stored, the television camera 20 is directed froma top plan view to generate scene video data of successive aircraftbeing produced. The output from television camera 20 is observed on amonitor 24 connected thereto by wire 22. The scene video of theproduction aircraft is also input to the frame grabber 32. The windowselector 26 is appropriately operated, either manually or by computercontrol, to generate an observation window of a particular edgeundergoing inspection which, for the purpose of this illustration, willbe the edge 13 of the forward wing of the aircraft 10. The output 34from the frame grabber 32 represents masked video of this edge portionwhich is input to the transform processor 36 that accomplishes an edgetransformation. The transformed data is then input to the processor 40,connected by wire 38 to the processor 36. This results in the generationof a test edge image.

At the bottom of FIG. 1, reference numeral 41 indicates a typicalreference edge image which has been pre-prestored. The image of the testedge portion 16 is indicated by reference numeral 51. The gist of thepresent invention is that, by combining the reference edge and test edgeimages 41 and 51, a combined edge moire display may be created ondisplay 52 by logically ORing the two images on the display 52. Themoire display provides visual information as to the angle between thetest edge and the reference edge. This will be explained in greaterdetail below.

FIGS. 2 and 3 indicate typical reference and test images, respectively,resulting from displays making the angle φ to each other.

FIG. 4 is an enlarged illustration of a moire interference patternproduced by combining reference and test images of FIGS. 2 and 3.

As a practical example, the system used in tests of this inventionemploys binarized edge replication to obtain edge transforms. Theselected images of the reference and test objects are in this waytransformed and overlaid to obtain in situ alignment of component partsduring assembly. This is accomplished by logically summing, in acomputer memory, the two transformed images to produce a moire alignmentpattern. The moire fringes are then analyzed to obtain a quantitativemeasure of angular alignment between the reference and test images.

Such edge replication is accomplished in image processor means providedfor transforming the edge or line into a replicated pattern of equallyspaced parallel contour lines. The processor consists of a suitabletransform processor which takes the single binarized line representingan edge and forms a series of pixel addresses representing each of aplurality of equally spaced points through which each line must pass.The binarized edge then replicated to form an array of equally spacedparallel lines parallel to the binarized edge to develop the patternshown in FIGS. 3 and 4. In one application, the above processor wasimplemented with a application macro within the NIH Image softwareprogram and could be executed in less than half a minute.

FIGS. 2 and 3 show two simulated edge transform patterns so derived froma fuselage center line (reference) and leading wing edge (test) and theformation of a corresponding moire pattern. The spacings for eachpattern can be the same value or can differ, depending upon the scale ofthe transforms. For this example, the spacings have been set equal toprovide a basic representation of moire effect.

Note that these fringe patterns would exactly parallel the physicaledges of the fuselage and wing sections. The angular difference betweenthe test and reference transforms is given by the angle φ. As the twoedge patterns come close to alignment, φ becomes quite small, and themoire fringe width D becomes correspondingly larger. In effect, smallmisalignments are magnified by the large moire fringe spacing, thusproviding sensitivity in the measurement where it is needed.

In order to understand the alignment geometry in greater detail,reference is made to FIG. 5 and 6. This is a simplified geometric viewof the overlapping test and reference images as shown in FIG. 4.

D is equal to the moire fringe spacing while S is the spacing betweenreplicated lines, which, for this analysis, is assumed to be the samefor each pattern. The angle φ is the angle between test and referencefringe patterns as well as the angle between the test edge 13 and thederived reference line 14. A greatly simplified analysis shows thetrigonometric relationship between these quantities is, for small φ:

    S/D≅tanφ

which, for small φ, can be shown to be satisfactorily approximated by:

    S/D≈φ, for small angles.

Solving for D, one finds:

    D≈S/φ.

A detailed analysis yields the same result.

From these trigonometric relationships, it is seen that as alignment isapproached between the structures, as seen in the reference and testimages, the angle φ goes to zero, and the moire fringe spacing D getslarger without limit, and at maximum alignment accuracy, D becomes andexceeds the width of the display. The effective magnification of thisalignment is the ratio S/φ which becomes quite large as alignment isachieved (i.e., φ→0).

It should be noted that in the above discussion the fuselage referenceline was relied upon as the reference edge for establishing a referenceedge image, but any edge or surface of the aircraft may serve the samepurpose. It is also to be emphasized that, although the presentinvention is discussed in connection with checking alignment ofproduction aircraft, the invention may be extended generally to checkingalignment between various linear formations of any structure (e.g.painted stripes, joints, borders, etc.). Further, it should be notedthat the number and shapes of a structure's edge can vary in curvatureand that line curves can be substituted for the straight curves shown.

The system of FIG. 1, as shown, will produce one measure per operation.Thus, in order to perform this operation for a multiple test sequence ofsuch operations, operator must manually and individually select eachmoire processing operation upon completion of the last.

Automated System

FIG. 7 shows a block diagram of a second embodiment of the inventiondesigned to automate this task, i.e., an automatic test template system.In addition to the elements camera 112, computer 114, and display 116also shown in FIG. 1, automatic template operations have been added. Anautomatic template processor 120 is in two-way communication with moireprocessing computer 114 and template memory module 124. Automatictemplate processor 120 outputs a template 126 that is either prescribedby operator 118 on line, or can be called up from a test librarycontained in template memory module 124. Operator 118 can prescribetemplate 126 by using light pen/mouse 122. Light pen/mouse 122 isconnected to automatic template processor 120 and is used by operator 18to highlight the areas where edges are to be measured on display 116.Once this plan is complete, the system can then be instructed toautomatically sequence through the area edges designated. The resultscan subsequently be viewed by operator 118 on display 116 and/or beprinted out as a documented test for the measurements prescribed by saidoperator and as automated by template 126.

The automatic test template architecture employs image processing andlogic operations to isolate and identify selected edges that can then betransferred to the moire edge alignment system for moire alignmentoperations.

FIG. 8 shows the basic image processing operation. A test image, asviewed by the high resolution video camera 112, shown in FIGS. 1 and 7,is thresholded by threshold circuit 130, of any suitable known type. Thethresholded test image is then binarized by binarizing circuit 132 ofany suitable known type. Binarizing of the thresholded test image setsall levels below the threshold to zeros and those above the thresholdlevel to ones. The binarized image is then edged by edging circuit 134,of any suitable known type, to produce a processed image. Edging circuit134 can be a form of a high pass filtering circuit, or other suitableprocessing means. Stated mathematically, where the image istwo-dimensional, with z denoting the intensity or video level:

PI(x,y)=I(x,y,z) Thld(z) BIN! EDGE where PI(x,y)=the processed image,I(x,y,z)=the test image, Thld(z)=threshold operation, BIN=binarizeoperation and EDGE=edge operation.

The processed image, PI(x,y), is then ready to be operated on by theautomatic template. The template is given, mathematically, by:

T(x,y)=NOT I(x,y,z)·LP(x,y)·Thld(z) BIN! where T(x,y)=display templatefield, I(x,y,z)=test image and LP(x,y)=light pen markup.

FIG. 9 shows the operation of producing a display template field. Theoperator, using a light pen markers up the test image. That is, theoperator highlights the selected areas for measurement using a device140. Device 140 can be either a light pen, or the like. Once the testimage is marked up, the marked up image is thresholded by thresholdcircuit 142, of any suitable known type. The marked up image thresholdedis then binarized by binarizing circuit 144, of any suitable known type.The binarized marked up image is then inverted by NOT gate 146, of anysuitable known type, to produce the template field of the displaytemplate.

The results of the operations shown in FIGS. 8 and 9 are combined, viaAND gate 150, shown in FIG. 5, to obtain the selected edges foralignment. Stated mathematically:

    SE(x,y)=PI(x,y)·T(x,y)

where SE(s,y)=the selected edges and PI(x,y)=the processed imageobtained in FIG. 8 and T(x,y)=the display template field obtained inFIG. 9.

Under changing conditions, and when new structures are placed undertest, a new template can be made. When it is required that existing testmodules be used from memory, the operator would be able to rotate andtranslate these templates first to nominally align them with referencepoints on the structure after which the test would proceed as previouslydescribed.

Application to Aircraft Alignment

The aircraft assembly under test is viewed by a TV camera mounted at theceiling of the manufacturing facility to obtain an overhead view of thestation where the structures are in final assembly, say, for example thewing to the fuselage.

To initialize the system, the operator selects the desired airplanereference surfaces and/or edges from the display console. Views of theairplane are then imaged by the TV camera and digitally stored in aresident computer on hard disk, RAM, or other suitable memory. Theoperator can select and isolate different sections of the aircraft forthe purpose of checking and aligning these subassemblies, as required.

The imaging system can be aligned along any prescribed axis such as theFRL or transverse axes of the aircraft. The selected images are viewedby the operator on the console monitor where programmable cursors,fiducial markers, and numerical readouts are provided to perform themoire measurements. The operator selects the test edge by placing awindow over the area of interest. The windowed image is thenthresholded, setting it above the noise level to extract a clean imagethat is then binarized and edge transformed.

The corresponding area of the selected reference image is processed inthe same way, producing an edge transform pattern. The twoedge-transformed images are then combined on the console display forminga moire pattern like that shown previously in FIG. 1. As the alignmentprocess proceeds, the test surface orientation is adjusted and the moirepattern changes. The operator will be able to conclude the alignment forthat surface when the moire fringe spacing fills the entire display orwhen it reaches a predetermined alignment width.

While several embodiments of the present invention have been shown anddescribed, it is to be understood that many changes and modificationsmay be made thereunto without departing from the spirit and scope of theinvention as defined in the appended claims.

What is claimed is:
 1. A method for aligning structures having linearfeatures, comprising the steps of:forming an optical image a portion ofa reference linear formation of a first structure; generating a videosignal reference line of an edge of the reference formation; generatinga transform signal of the reference edge; converting the transformsignal to a binarized image; replicating said binarized image into a setof spaced parallel lines parallel to said binarized reference edge;forming an optical image a portion of a reference linear formation of asecond structure; generating a video signal reference line of an edge ofthe test formation from said second structure; generating a transformsignal of a test edge; converting the transform signal to a binarizedtest image; replicating said binarized test image into a set of spacedparallel lines parallel to said binarized test edge; combining thereplicated, binarized images of the reference and test edges to form amoire pattern in which fringe spacing (D) is approximately equal to thespacing (S) between reference and test edges divided by the angle (φ)between them; and moving the structures until the spacing expands to alimit, at which condition said structures are aligned.
 2. The method setforth in claim 1 together with the step of displaying the moire pattern,which is indicative of the angle between the test and reference linearformations.
 3. The method set forth in claim 1 together with the step ofstoring the generated video signal of the reference linear formation. 4.The method set forth in claim 1 together with the step of selecting theobservation window along any portion of the test linear formation. 5.The method set forth in claim 1 wherein the structure is an aircraftundergoing production.
 6. Apparatus for aligning structures havinglinear features, comprising:means for forming an optical image a portionof a reference linear formation of a first structure; means forgenerating a video signal reference line of an edge of the referenceformation; means for generating a transform signal of the referenceedge; means for converting the transform signal to a binarized image;means for replicating said binarized image into a set of spaced parallellines parallel to said binarized reference edge; means for forming anoptical image a portion of a reference linear formation of a secondstructure; means for generating a video signal reference line of an edgeof the test formation from said second structure; means for generating atransform signal of the test edge; means for converting the transformsignal to a binarized test image; means for replicating said binarizedtest image into a set of spaced parallel lines parallel to saidbinarized test edge; and means for combining the replicated, binarizedimages of the reference and test edges to form a moire pattern in whichfringe spacing (D) is approximately equal to the spacing (S) betweenreference and test edges divided by the angle (φ) between them so thatas said structures are moved to increase the spacing of said fringes,they become progressively aligned.
 7. The system set forth in claim 6wherein the optical imaging means comprises;a monitor connected to theoutput of the viewing means; means connected to the output of themonitor means for viewing a structure, incorporating the linearformations; and means for variably selecting the position of a windowsuperimposed on the structure to effectively mask parts of the structureextraneous to the test linear formation.
 8. The system set forth inclaim 6 wherein the combining means comprises a display.
 9. The systemset forth in claim 6 wherein the viewing means in a high resolutionvideo camera.